Ranking hospitals based on preventable hospital death rates:a systematic review with implications for both direct measurement and indirect measurement through standardized mortality rates

Objectives There is interest in monitoring avoidable or preventable deaths measured directly or indirectly through standardized mortality rates (SMRs). We reviewed studies that use implicit case note reviews to estimate the range of preventable death rates observed, the measurement characteristics of those estimates, and the measurement procedures used to generate them. We comment on the implications for monitoring SMRs and illustrate a way to calculate the number of reviews needed to establish a reliable estimate of preventability of one death or the hospital preventable death rate. Design Systematic review of the literature supplemented by re-analysis of authors previously published and un-published data and measurement design calculations. Data source Searches in PubMed, MEDLINE (OvidSP) and Web of Knowledge in June 2012, updated December 2017. Eligibility criteria Studies of hospital-wide admissions from general and acute medical wards where preventable deaths rates are provided or can be estimate and which can provide inter- observer variations. Results Twenty-four studies were included from 1983-2017. Recent larger studies suggest consistently low rates of preventable deaths (3.0-6.5% since 2012). Reliability of a single review for distinguishing between individual cases with regard to the preventability of death had a Kappa rate of 0.27-0.50 for deaths and 0.24-0.76 for adverse events. A Kappa of 0.35 would require an average of 8-17 reviews of a single case to be precise enough to have confidence about high stakes decisions to change care procedures or impose sanctions within a hospital as a result. No study estimated the variation in preventable deaths across hospitals, although we were able to re-analyse one study to obtain an estimate. Based on this estimate, 200-300 total case-note reviews per hospital could be required to reliably distinguish between hospitals. The studies display considerable heterogeneity: 13/24 studies defined preventable with a threshold of ≥4 in a six-category Likert scale; 11/24 involved a two-stage screening process with nurses at the first stage and physicians at the second. Fifteen studies provided expert clinical review support for reviewer disagreements, advice, or quality control. A ‘generalist/internist’ was the modal physician specialty for reviewers and they received 1-3 days of generic tools orientation and case-note review practice. Methods did not consider the influence of human or environmental factors. Conclusions The literature provides limited information about the measurement characteristics of preventable deaths that suggests substantial numbers of reviews may be needed to create reliable estimates of preventable deaths at the individual or hospital level. Any operational program would require population specific estimates of reliability. Preventable death rates are low, which is likely to make it difficult to use SMRs based on all deaths to validly profile hospitals. The literature provides little information to guide improvements in the measurement procedures. Systematic review registration The systematic review was conceived prior to PROSPERO, and so has not been registered

Organizational factors and depression management in community-based primary care settings

Abstract Background Evidence-based quality improvement models for depression have not been fully implemented in routine primary care settings. To date, few studies have examined the organizational factors associated with depression management in real-world primary care practice. To successfully implement quality improvement models for depression, there must be a better understanding of the relevant organizational structure and processes of the primary care setting. The objective of this study is to describe these organizational features of routine primary care practice, and the organization of depression care, using survey questions derived from an evidence-based framework. Methods We used this framework to implement a survey of 27 practices comprised of 49 unique offices within a large primary care practice network in western Pennsylvania. Survey questions addressed practice structure (e.g., human resources, leadership, information technology (IT) infrastructure, and external incentives) and process features (e.g., staff performance, degree of integrated depression care, and IT performance). Results The results of our survey demonstrated substantial variation across the practice network of organizational factors pertinent to implementation of evidence-based depression management. Notably, quality improvement capability and IT infrastructure were widespread, but specific application to depression care differed between practices, as did coordination and communication tasks surrounding depression treatment. Conclusions The primary care practices in the network that we surveyed are at differing stages in their organization and implementation of evidence-based depression management. Practical surveys such as this may serve to better direct implementation of these quality improvement strategies for depression by improving understanding of the organizational barriers and facilitators that exist within both practices and practice networks. In addition, survey information can inform efforts of individual primary care practices in customizing intervention strategies to improve depression management.http://deepblue.lib.umich.edu/bitstream/2027.42/78269/1/1748-5908-4-84.xmlhttp://deepblue.lib.umich.edu/bitstream/2027.42/78269/2/1748-5908-4-84-S1.PDFhttp://deepblue.lib.umich.edu/bitstream/2027.42/78269/3/1748-5908-4-84.pdfPeer Reviewe

Design of a continuous quality improvement program to prevent falls among community-dwelling older adults in an integrated healthcare system

<p>Abstract</p> <p>Background</p> <p>Implementing quality improvement programs that require behavior change on the part of health care professionals and patients has proven difficult in routine care. Significant randomized trial evidence supports creating fall prevention programs for community-dwelling older adults, but adoption in routine care has been limited. Nationally-collected data indicated that our local facility could improve its performance on fall prevention in community-dwelling older people. We sought to develop a sustainable local fall prevention program, using theory to guide program development.</p> <p>Methods</p> <p>We planned program development to include important stakeholders within our organization. The theory-derived plan consisted of 1) an initial leadership meeting to agree on whether creating a fall prevention program was a priority for the organization, 2) focus groups with patients and health care professionals to develop ideas for the program, 3) monthly workgroup meetings with representatives from key departments to develop a blueprint for the program, 4) a second leadership meeting to confirm that the blueprint developed by the workgroup was satisfactory, and also to solicit feedback on ideas for program refinement.</p> <p>Results</p> <p>The leadership and workgroup meetings occurred as planned and led to the development of a functional program. The focus groups did not occur as planned, mainly due to the complexity of obtaining research approval for focus groups. The fall prevention program uses an existing telephonic nurse advice line to 1) place outgoing calls to patients at high fall risk, 2) assess these patients' risk factors for falls, and 3) triage these patients to the appropriate services. The workgroup continues to meet monthly to monitor the progress of the program and improve it.</p> <p>Conclusion</p> <p>A theory-driven program development process has resulted in the successful initial implementation of a fall prevention program.</p

Assessing Quality of Care of Elderly Patients Using the ACOVE Quality Indicator Set: A Systematic Review

Background: Care of the elderly is recognized as an increasingly important segment of health care. The Assessing Care Of Vulnerable Elderly (ACOVE) quality indicators (QIs) were developed to assess and improve the care of elderly patients. Objectives: The purpose of this review is to summarize studies that assess the quality of care using QIs from or based on ACOVE, in order to evaluate the state of quality of care for the reported conditions. Methods: We systematically searched MEDLINE, EMBASE and CINAHL for English-language studies indexed by February 2010. Articles were included if they used any ACOVE QIs, or adaptations thereof, for assessing the quality of care. Included studies were analyzed and relevant information was extracted. We summarized the results of these studies, and when possible generated an overall conclusion about the quality of care as measured by ACOVE for each condition, in various settings, and for each QI. Results: Seventeen studies were included with 278 QIs (original, adapted or newly developed). The quality scores showed large variation between and within conditions. Only a few conditions showed a stable pass rate range over multiple studies. Overall, pass rates for dementia (interquartile range (IQR): 11%-35%), depression (IQR: 27%-41%), osteoporosis (IQR: 34%-43%) and osteoarthritis (IQR: 29-41%) were notably low. Medication management and use (range: 81%-90%), hearing loss (77%-79%) and continuity of care (76%-80%) scored higher than other conditions. Out of the 278 QIs, 141 (50%) had mean pass rates below 50% and 121 QIs (44%) had pass rates above 50%. Twenty-three percent of the QIs scored above 75%, and 16% scored below 25%. Conclusions: Quality of care per condition varies markedly across studies. Although there has been much effort in improving the care for elderly patients in the last years, the reported quality of care according to the ACOVE indicators is still relatively lo

EQUIP: Implementing chronic care principles and applying formative evaluation methods to improve care for schizophrenia: QUERI Series

<p>Abstract</p> <p>Background</p> <p>This paper presents a case study that demonstrates the evolution of a project entitled "Enhancing QUality-of-care In Psychosis" (EQUIP) that began approximately when the U.S. Department of Veterans Affairs' Quality Enhancement Research Initiative (QUERI), and implementation science were emerging. EQUIP developed methods and tools to implement chronic illness care principles in the treatment of schizophrenia, and evaluated this implementation using a small-scale controlled trial. The next iteration of the project, EQUIP-2, was further informed by implementation science and the use of QUERI tools.</p> <p>Methods</p> <p>This paper reports the background, development, results and implications of EQUIP, and also describes ongoing work in the second phase of the project (EQUIP-2). The EQUIP intervention uses implementation strategies and tools to increase the adoption and implementation of chronic illness care principles. In EQUIP-2, these strategies and tools are conceptually grounded in a stages-of-change model, and include clinical and delivery system interventions and adoption/implementation tools. Formative evaluation occurs in conjunction with the intervention, and includes developmental, progress-focused, implementation-focused, and interpretive evaluation.</p> <p>Results</p> <p>Evaluation of EQUIP provided an understanding of quality gaps <it>and </it>how to address related problems in schizophrenia. EQUIP showed that solutions to quality problems in schizophrenia differ by treatment domain and are exacerbated by a lack of awareness of evidence-based practices. EQUIP also showed that improving care requires creating resources for physicians to help them easily implement practice changes, plus intensive education as well as product champions who help physicians use these resources. Organizational changes, such as the addition of care managers and informatics systems, were shown to help physicians with identifying problems, making referrals, and monitoring follow-up. In EQUIP-2, which is currently in progress, these initial findings were used to develop a more comprehensive approach to implementing and evaluating the chronic illness care model.</p> <p>Discussion</p> <p>In QUERI, small-scale projects contribute to the development and enhancement of hands-on, action-oriented service-directed projects that are grounded in current implementation science. This project supports the concept that QUERI tools can be useful in implementing complex care models oriented toward evidence-based improvement of clinical care.</p

A new implicit review instrument for measuring quality of care delivered to pediatric patients in the emergency department

BackgroundThere are few outcomes experienced by children receiving care in the Emergency Department (ED) that are amenable to measuring for the purposes of assessing of quality of care. The purpose of this study was to develop, test, and validate a new implicit review instrument that measures quality of care delivered to children in EDs.MethodsWe developed a 7-point structured implicit review instrument that encompasses four aspects of care, including the physician's initial data gathering, integration of information and development of appropriate diagnoses; initial treatment plan and orders; and plan for disposition and follow-up. Two pediatric emergency medicine physicians applied the 5-item instrument to children presenting in the highest triage category to four rural EDs, and we assessed the reliability of the average summary scores (possible range of 5-35) across the two reviewers using standard measures. We also validated the instrument by comparing this mean summary score between those with and without medication errors (ascertained independently by two pharmacists) using a two-sample t-test.ResultsWe reviewed the medical records of 178 pediatric patients for the study. The mean and median summary score for this cohort of patients were 27.4 and 28.5, respectively. Internal consistency was high (Cronbach's alpha of 0.92 and 0.89). All items showed a significant (p &lt; 0.005) positive correlation between reviewers using the Spearman rank correlation (range 0.24 to 0.39). Exact agreement on individual items between reviewers ranged from 70.2% to 85.4%. The Intra-class Correlation Coefficient for the mean of the total summary score across the two reviewers was 0.65. The validity of the instrument was supported by the finding of a higher score for children without medication errors compared to those with medication errors which trended toward significance (mean score = 28.5 vs. 26.0, p = 0.076).ConclusionThe instrument we developed to measure quality of care provided to children in the ED has high internal consistency, fair to good inter-rater reliability and inter-rater correlation, and high content validity. The validity of the instrument is supported by the fact that the instrument's average summary score was lower in the presence of medication errors, which trended towards statistical significance

Identifying quality improvement intervention publications - A comparison of electronic search strategies

Abstract Background The evidence base for quality improvement (QI) interventions is expanding rapidly. The diversity of the initiatives and the inconsistency in labeling these as QI interventions makes it challenging for researchers, policymakers, and QI practitioners to access the literature systematically and to identify relevant publications. Methods We evaluated search strategies developed for MEDLINE (Ovid) and PubMed based on free text words, Medical subject headings (MeSH), QI intervention components, continuous quality improvement (CQI) methods, and combinations of the strategies. Three sets of pertinent QI intervention publications were used for validation. Two independent expert reviewers screened publications for relevance. We compared the yield, recall rate, and precision of the search strategies for the identification of QI publications and for a subset of empirical studies on effects of QI interventions. Results The search yields ranged from 2,221 to 216,167 publications. Mean recall rates for reference publications ranged from 5% to 53% for strategies with yields of 50,000 publications or fewer. The 'best case' strategy, a simple text word search with high face validity ('quality' AND 'improv*' AND 'intervention*') identified 44%, 24%, and 62% of influential intervention articles selected by Agency for Healthcare Research and Quality (AHRQ) experts, a set of exemplar articles provided by members of the Standards for Quality Improvement Reporting Excellence (SQUIRE) group, and a sample from the Cochrane Effective Practice and Organization of Care Group (EPOC) register of studies, respectively. We applied the search strategy to a PubMed search for articles published in 10 pertinent journals in a three-year period which retrieved 183 publications. Among these, 67% were deemed relevant to QI by at least one of two independent raters. Forty percent were classified as empirical studies reporting on a QI intervention. Conclusions The presented search terms and operating characteristics can be used to guide the identification of QI intervention publications. Even with extensive iterative development, we achieved only moderate recall rates of reference publications. Consensus development on QI reporting and initiatives to develop QI-relevant MeSH terms are urgently needed

Lessons for non-VA care delivery systems from the U.S. Department of Veterans Affairs Quality Enhancement Research Initiative: QUERI Series

The U.S. Veterans Health Administration (VHA) may have a very different structure and function from the organizations and practices that provide medical care to most Americans, but those organizations and practices could learn a lot from the VHA's Quality Enhancement Research Initiative (QUERI). There are at least six topics of increasing importance for implementation research where QUERI experience should be of value to other non-VHA organizations, both within and external to the United States: 1) Researcher-clinical leader partnerships for care improvement; 2) Attention to culture, capacity, leadership, and a supportive infrastructure; 3) Practical economic evaluation of quality implementation efforts; 4) Human subject protection problems; 5) Sustainability of improvements; and 6) Scale-up and spread of improvements

Profiling quality of care: Is there a role for peer review?

BACKGROUND: We sought to develop a more reliable structured implicit chart review instrument for use in assessing the quality of care for chronic disease and to examine if ratings are more reliable for conditions in which the evidence base for practice is more developed. METHODS: We conducted a reliability study in a cohort with patient records including both outpatient and inpatient care as the objects of measurement. We developed a structured implicit review instrument to assess the quality of care over one year of treatment. 12 reviewers conducted a total of 496 reviews of 70 patient records selected from 26 VA clinical sites in two regions of the country. Each patient had between one and four conditions specified as having a highly developed evidence base (diabetes and hypertension) or a less developed evidence base (chronic obstructive pulmonary disease or a collection of acute conditions). Multilevel analysis that accounts for the nested and cross-classified structure of the data was used to estimate the signal and noise components of the measurement of quality and the reliability of implicit review. RESULTS: For COPD and a collection of acute conditions the reliability of a single physician review was quite low (intra-class correlation = 0.16–0.26) but comparable to most previously published estimates for the use of this method in inpatient settings. However, for diabetes and hypertension the reliability is significantly higher at 0.46. The higher reliability is a result of the reviewers collectively being able to distinguish more differences in the quality of care between patients (p < 0.007) and not due to less random noise or individual reviewer bias in the measurement. For these conditions the level of true quality (i.e. the rating of quality of care that would result from the full population of physician reviewers reviewing a record) varied from poor to good across patients. CONCLUSIONS: For conditions with a well-developed quality of care evidence base, such as hypertension and diabetes, a single structured implicit review to assess the quality of care over a period of time is moderately reliable. This method could be a reasonable complement or alternative to explicit indicator approaches for assessing and comparing quality of care. Structured implicit review, like explicit quality measures, must be used more cautiously for illnesses for which the evidence base is less well developed, such as COPD and acute, short-course illnesses